Information processing device, information processing server, information processing method, information extracting method and program

ABSTRACT

According to an illustrative embodiment an information processing system is provided. The system includes a processor for determining one or more candidate tags based on input data, the candidate tags being included within a hierarchical structure; and a display for displaying the candidate tags in a manner indicative of the candidate tags&#39; positions in the hierarchical structure.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. JP 2011-222538 filed in the Japanese Patent Office on Nov. 7, 2011,the entire content of which is hereby incorporated by reference herein.

BACKGROUND

The present disclosure relates to an information processing device, aninformation processing server, an information processing method, aninformation extracting method, and a program.

There is a technique called clustering for creating a group of datawhich are positioned in a close distance within a feature spaceprescribed by a predetermined feature quantity, and the technique iswidely applied to various fields. Also, there is a technique widely usedto create a tree-like data structure by grouping data included inclusters generated by the clustering.

The data structure, which is thus created, has a configuration to havesuch structure that an upper hierarchical level includes a lowerhierarchical level. Therefore, the data structure is used for searchingfor desired data by selecting from a group having coarser granularity toa group having finer granularity in order. Also, the data structure isused to create new groups of certain data each having differentgranularity by changing the hierarchical level (refer to, for example,Japanese Patent Application Laid-Open Publication No. 2007-122562).

When searching for a data group, many users trace in order from the topthe hierarchy structure which is formed by clustering technique toobtain desired data. The Japanese Patent Application Laid-OpenPublication No. 2007-122562 teaches a technique to provide a displayscreen which allows users to instinctively comprehend a hierarchystructure and provide easy data search.

SUMMARY

Here, a case where a user performs an operation to newly associate datawith a group having been created by using the technique as in JapanesePatent Application Laid-Open Publication No. 2007-122562 (e.g. a case ofnewly associating an image contents with data structure of a treestructure related to image contents) will be considered. In such a case,the user determines the group to which the data to be newly processedcorresponds, and repeats operations to scroll a display screen and clickan input device such as a mouse until a hierarchical level in which therelevant group exists is displayed on the display screen. Due to this,in the case of newly associating data with a group included in the datastructure of the tree structure that is already created, there has beena room for improvement in operability of applications.

Thus, in view of the above circumstances, in the present disclosure, aninformation processing device, information processing server,information processing method, information extracting method, andprogram therefor that are capable of further improving convenience ofthe user operation are proposed.

A information processing system according to an illustrative embodimentincludes a processor for determining one or more candidate tags based oninput data, the candidate tags being included within a hierarchicalstructure; and a display for displaying the candidate tags in a mannerindicative of the candidate tags' positions in the hierarchicalstructure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram showing a tagging process with respectto data;

FIG. 2 is an explanatory diagram schematically showing an overallconfiguration of information processing system according to anembodiment 1 of the present disclosure;

FIG. 3 is an explanatory diagram for explaining a tree structure;

FIG. 4 is an explanatory diagram showing an example of the informationprocessing system according to the embodiment 1;

FIG. 5 is a block diagram showing an example of a configuration of aninformation processing server according to the embodiment 1;

FIG. 6 is an explanatory diagram showing an example of the treestructure according to the embodiment 1;

FIG. 7A is an explanatory diagram showing an example of a tag candidateextracting process according to the embodiment 1;

FIG. 7B is an explanatory diagram showing an example of the tagcandidate extracting process according to the embodiment 1;

FIG. 7C is an explanatory diagram showing an example of the tagcandidate extracting process according to the embodiment 1;

FIG. 8 is an explanatory diagram showing a tag candidate interpolatingprocess according to the embodiment 1;

FIG. 9 is a block diagram showing an example of a configuration of aninformation processing device according to the embodiment 1;

FIG. 10 is an explanatory diagram showing an example of a displaycontrolling process according to the embodiment 1;

FIG. 11 is an explanatory diagram showing an example of the displaycontrolling process according to the embodiment 1;

FIG. 12 is an explanatory diagram showing an example of the displaycontrolling process according to the embodiment 1;

FIG. 13 is an explanatory diagram showing an example of the displaycontrolling process according to the embodiment 1;

FIG. 14 is an explanatory diagram showing an example of the displaycontrolling process according to the embodiment 1;

FIG. 15 is an explanatory diagram showing an example of the displaycontrolling process according to the embodiment 1;

FIG. 16 is an explanatory diagram showing another example of the tagcandidate extracting process and the display controlling processaccording to the embodiment 1;

FIG. 17 is an explanatory diagram showing another example of the tagcandidate extracting process and the display controlling according tothe embodiment 1;

FIG. 18 is an explanatory diagram showing another example of the tagcandidate extracting process and the display controlling according tothe embodiment 1;

FIG. 19 is a flow chart showing an example of a flow of an informationextracting method and an information processing method according to theembodiment 1;

FIG. 20 is a block diagram showing a variant of the informationprocessing system according to the embodiment 1; and

FIG. 21 is a block diagram showing an example of a hardwareconfiguration of the information processing server according to anembodiment disclosed herein.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Note that, explanations will be given in the following order.

(1) As to tagging process

(2) As to concept of tree structure

(3) Embodiment 1

-   -   (3-1) As to information processing system    -   (3-2) As to configuration of information processing server    -   (3-3) As to configuration of information processing device    -   (3-4) As to another example related to input data    -   (3-5) As to information extracting method and information        processing method    -   (3-6) Variant

(4) As to hardware configuration of information processing server andinformation processing device according to the embodiment of the presentdisclosure

(5) Conclusion

(As to Tagging Process)

Prior to giving the explanation according to the embodiment of thepresent disclosure, a result of consideration given by the presentinventor regarding a tagging process will briefly be explained withreference to FIG. 1, and the tagging process implemented by theembodiment of the present disclosure will briefly be explained withreference to FIG. 2.

In recent years, services that perform various types of healthmanagement based on inputs of meal logs by a user are increasing. Insuch services, detailed meal logs are required so as to perform thehealth management based on accurate data. However, in order to do so,the user is required to input food that the user him/herself ate fromamong a vast variety, namely hundreds and thousands, of food, andcomplication of this procedure is being a problem.

As one technique to resolve this complication, a method of providingfood tag candidates based on a result of food identification on foodimages, or a history of past tagging results of the user may beconsidered.

However, even in a case where the food tags are indicated in a treestructure e.g. as shown in FIG. 1, the user selects a food tag (rices,noodles, and the like) from an upper hierarchical level toward a lowerhierarchical level, and repeats processes while scrolling a displaywindow or a display screen until reaching an aimed hierarchical level.

Such a repetition of operations fails convenience of the user, so atagging method with satisfactory operability is desired. Thus, thepresent inventor eagerly investigated a method capable of improving theoperability of applications and newly associating data easily with oneof a plurality of tags indicated in a tree structure (in the example ofFIG. 1, a method of newly associating a food with one of a plurality offood tags existing in the tree structure related to food tags).

As a result, as shown in FIG. 2, a method has been conceived in which,based on input data related to the food to be associated (e.g. imagedata in which the food to be associated is photographed), a tag relevantto the input data is identified, and a food tag that is highly likely tobe selected by the user is narrowed down and presented based on anidentification result of the tag.

Further, a method of presenting a tag candidate which the presentinventor has conceived is applicable not only to the case of associatingthe image data, in which food is an object, with a tree structure inwhich food tags are structured, but also to cases in which voluntaryinput data is newly associated with a tree structure, such asassociating a new and event or concept that is related to the treestructure indicating an inclusion relation of events and concepts.

Hereinbelow, the method of presenting a tag candidate that the presentinventor has conceived will be explained in detail.

(As to Concept of Tree Structure)

Prior to giving explanations according to the embodiments of the presentdisclosure, terminology of the tree structure will briefly be explainedwith reference to FIG. 3. FIG. 3 is an explanatory diagram forexplaining the tree structure.

The tree structure includes a plurality of elements (marked with acircle in FIG. 3) as shown in FIG. 3. Each of the plurality of elementsis referred to as node. In the tree structure, a node positioned at thetop is referred to as route node. As viewed from the route node, severalbranches extend downward from the route node in the figure, and at theend of each branch, a node is positioned respectively. By repeatingbranching as described above, the tree structure is formed to have amultilayered structure as shown in FIG. 3. In the tree structure, a nodepositioned at the bottom is referred to as a leaf node. As illustratedin the figure, no branch extends from the leaf nodes.

Here, when focusing to a node “B” shown in FIG. 3, a branch extendingupward from the node B is connected to a route node; and branchesextending downward from the node B are connected to two nodes (leafnodes) of a leaf 3 and a leaf 4. Herein, a node, which is directlyconnected to a branch extending upward (i.e. toward the route node) likethe route node with respect to the node B, will be referred to as parentnode. Also, a node, which is directly connected to a branch extendingdownward (i.e. in a direction opposite to the direction toward routenode) with respect to the node B like the leaf 3 and the leaf 4, will bereferred to as child node.

Naming of “parent node” and “child node” is just a relative naming. Whenfocused node is changed, the naming is also changed. For example, thenode B is the parent node with respect to the leaf 3 or leaf 4, but thenode B is a child node with respect to the route node.

The tree structure has a multilayered structure as shown in FIG. 3.Hereinafter, a hierarchical level to which the route node belongs willbe referred to as 0-th hierarchical level; a hierarchical level to whicha child node of the route node belongs will be called as firsthierarchical level; and a hierarchical level to which a child node ofthe node which is positioned at the first hierarchical level will bereferred to as second hierarchical level. Hereinafter, the hierarchieswill be referred to as a third hierarchical level, a fourth hierarchicallevel . . . in order as necessary.

When focusing to the node B, child nodes other than the focused node,which are branched from the parent node of a certain node like the nodeA and node C, will be referred to as sibling node. For example, whenfocusing to the leaf 3 in FIG. 3, a sibling node thereof is the leaf 4.

FIG. 3 shows an example of a case where a plurality of branches extendfrom a certain node. However, the number of the branches extendingdownward (i.e. in a direction opposite to the direction toward routenode) may be only one. Also, the number of the branches extending from acertain node is not limited to the example shown in FIG. 3.

By using the tree structure as shown in FIG. 3, the inclusion relationof the events and concepts associated with respective nodes andrespective leaves can clearly be indicated.

Further, in the embodiments of the present disclosure explained below,not only the tree structure as shown in FIG. 3, but also, various typesof structured information that indicate the inclusion relation of theevents or concepts and that can be treated similar to the tree structureexemplified in FIG. 3 will be treated as information relevant to thetree structure indicating the inclusion relation of the events orconcepts.

Embodiment 1 <As to Information Processing System>

Firstly, an information processing system according to the embodiment 1of the present disclosure will briefly be explained with reference toFIG. 4. FIG. 4 is an explanatory diagram showing an informationprocessing system 1 according to the present embodiment.

As shown in FIG. 4, the information processing system 1 according to thepresent embodiment includes an information processing server 10 and aninformation processing device 20. Further, the information processingserver 10 and the information processing device 20 are configuredcapable of communicating with one another through a network.

The network is a communication line that connects the informationprocessing server 10 and the information processing device 20 to oneanother in a manner capable of bidirectional communication. This networkis configured for example of a public communication line network such asthe Internet, telephone communication line, satellite communicationnetwork, broadcast communication path and the like, or a leased linenetwork such as a WAN (Wide Area Network), LAN (Local Area Network),IP-VPN (Internet Protocol-Virtual Private Network), Ethernet (registeredtrademark), wireless LAN and the like, and it may be wired or wireless.That is, the information processing system 1 according to the presentembodiment may be a part of a public service using the publiccommunication line network such as the Internet and the like, or may bea private one using a home network utilizing the LAN and the like andthat is not made public to third parties.

The information processing server 10 extracts, for each of granularitiesthat is to be a reference of an inclusion relation of events orconcepts, a tag that may correspond to input data from among a pluralityof tags, based on the input data including at least one of a character,image or sound designated by the information processing device 20. Theplurality of tags indicates distinctions for classifying the events orconcepts that are respectively associated with the tree structurecorresponding to the inclusion relation of the events or concepts. Whenthe information processing server 10 extracts plural tags that areassumed to correspond to the designated input data, informationregarding the extracted tags are outputted to a relevant informationprocessing device 20. Further, when a tag corresponding to the inputdata is selected by the information processing device 20, the selectedtag is associated with the input data based on information regarding aresult of the tag selection. Due to this, the input data is associatedwith one of the tags in a target tree structure.

The information processing device 20 designates the input data includingat least one of the character, image or sound and to which tagging isdesired to the information processing server 10, and displays extractedtags on a display screen based on the information related to the tagsextracted by the information processing server 10. Further, when a tagcorresponding to the input data is selected by a user from the tagsdisplayed on the display screen, the information related to the resultof tag selection is outputted to the information processing server 10.

As an information processing device 20 as above, e.g. a personalcomputer, television, various recorders such as a DVD recorder andBlu-Ray recorder and the like, car navigation system, and informationappliance can be exemplified. Further, the information processing device20 may be one of various communication devices such as a cell phone,PDA, so-called smart phone and the like, portable contents player suchas a portable music player and the like, portable game machine, andtablet type portable information terminal having a touch panel and thelike.

The information processing server 10 and the information processingdevice 20 as above will again be explained in detail hereinbelow.

Note that, in FIG. 4, although there only is one information processingserver 10 shown, a number of the information processing server 10existing on the network is not limited to this example, and a pluralityof information processing servers 10 may exist on the network. Further,in FIG. 4, although three information processing devices 20 are shown,the information processing device 20 existing on the network may be lessthan three, or may be at or more than four.

<As to Configuration of Information Processing Server>

Next, a configuration of the information processing server 10 accordingto the present embodiment will be explained in detail with reference toFIG. 5 to FIG. 8. FIG. 5 is a block diagram showing an example of theconfiguration of the information processing server 10 according to thepresent embodiment. FIG. 6 is an explanatory diagram showing an exampleof the tree structure according to the present embodiment. FIG. 7A toFIG. 7C are explanatory diagrams showing examples of a tag candidateextracting process according to the present embodiment. FIG. 8 is anexplanatory diagram showing a tag candidate interpolating processaccording to the present embodiment.

Note that, in the below explanation, image data created by photographinga food will be exemplified as an example of input data including atleast one of a character, image or sound, and the explanation will begiven of a case in which food tag candidates corresponding to the foodthat is photographed are extracted from a tree structure related to foodtags based on the image data related to the aforementioned food.

As shown in FIG. 5, the information processing server 10 according tothe present embodiment primarily includes a data acquiring section 101,process target area detecting section 103, tag identifying section 105,tag candidate extracting section 107, tag candidate informationoutputting section 109, user operation information acquiring section111, tag allotting section 113, and storing section 115.

The data acquiring section 101 is implemented e.g. by a CPU (CentralProcessing Unit), ROM (Read Only Memory), RAM (Random Access Memory),input device, communicating device, and the like. The data acquiringsection 101 acquires the input data including at least one of acharacter, image or sound as designated by the information processingdevice 20 from the information processing device 20, or from variousinformation managing servers on the network with which the informationprocessing server 10 can communicate. Further, in a case where theinformation processing server 10 itself has a function of theinformation managing server to retain and manage a variety ofinformation, the data acquiring section 101 may acquire the input datadesignated by the information processing device 20 from a storing areasuch as the storing section 115.

When the designated input data is acquired from the various devicesmanaging the aforementioned data, the data acquiring section 101 outputsthe acquired input data to the process target area detecting section 103to be described later.

The process target area detecting section 103 that is an example of anarea detecting section is implemented e.g. by a CPU, ROM, RAM, and thelike. The process target area detecting section 103 detects a data area,to be used in a tag identifying process by the tag identifying section105 and a tag candidate extracting process by the tag candidateextracting section 107 as will be described later, from among the inputdata outputted from the data acquiring section 101.

In detecting a data area (process target area) that is to be a target ofprocessing in the input data, the process target area detecting section103 may determine a data area designated (manually) by a user operationas the process target area. Further, the process target area detectingsection 103 may automatically detect the process target area from theinput data by using a known processing technique such as a languagerecognition process, image detection/image recognition process, soundrecognition process and the like.

In a case e.g. where the input data is the image data related to a food,the process target area detecting section 103 can automatically detectthe process target area by a following method.

For example, by using a known technique to cut out a portion where acolor component or texture characteristic of a food exists from animage, the process target area can be detected from the image data inwhich the food is photographed. Alternatively, the process target areacan be detected from the image data in which the food is photographed byusing a known technique to cut out a portion having a characteristicshape such as a shape of a plate or dish on which the food is servedfrom the image.

By using the processes as above, the process target area detectingsection 103 can e.g. detect the portion having a shape of a circle,oval, or rectangle that is characteristic of the plate or dish afterhaving performed a contour detecting process on the image data that isthe input data, and determine the detected area as the portion where thefood exists (process target area).

When the process target area is detected from the input data as above,the process target area detecting section 103 creates informationindicating a position of the process target area. Thereafter, theprocess target area detecting section 103 associates the createdinformation indicating the position of the process target area with theinput data, and outputs the same to the tag identifying section 105 tobe described later. Further, the process target area detecting section103 may store variety of information related to the detected processtarget area as a history in the storing section 115 to be describedlater.

Note that, a case in which a plurality of process target areas isdetected in one piece of input data may occur. In such a case, theprocess target area detecting section 103 gives identificationinformation (e.g. labeling numbers and the like) that differ from oneanother to the respective one of the detected process target areas, andthereby distinguishes the plurality of detected process target area.Thereafter, each of the information indicating the positions of theprocess target areas is outputted to the tag identifying section 105 tobe described later.

The tag identifying section 105 is implemented e.g. by a CPU, ROM, RAM,and the like. The tag identifying section 105 identifies aninput-data-corresponding tag that is a tag corresponding to the inputdata based on the input data. Here, a tag is a distinction forclassifying events or concepts. In taking e.g. food as an example, namesof respective food used in classifying the food (curry and rice, beefsteak, sushi, and the like), or group names indicating types of food(meat cuisine, seafood cuisine, Japanese, Italian, and the like)correspond to tags.

More specifically, the tag identifying section 105 identifies a tagcorresponding to the process target area in the input data based onvarious identifying processes by using the input data and theinformation indicating the position of the process target area outputtedfrom the process target area detecting section 103.

In identifying the tag corresponding to the input data (hereinbelowreferred to as input-data-corresponding tag), the tag identifyingsection 105 can use any known technique, however, the tag identifyingsection 105 may identify the input data-corresponding-tag by e.g. usingmethods as shown below.

(a) An identifying process using a discriminator related to the inputdata

(b) An identifying process by searching similar data that is similar tothe input data

(c) An identifying process based on an association history of tags inthe tree structure

The identifying process using the discriminator related to the inputdata as indicated in the above (a) is a method to identify the tag ofthe input data by using the discriminator that is created by using amachine learning technique utilizing training data related to the inputdata. In using this identifying method, a plurality of data (trainingdata) in which e.g. an image in which the food is photographed (foodimage) and a name of the food image (food tag) are associated with oneanother is used to predeterminedly create discriminators related to therespective food, and the created discriminators are stored in thestoring section 115 and the like to be described later. Thereafter, thetag identifying section 105 identifies the food tag corresponding to thefood image by inputting the food image to the respective discriminators.

In a case of using the identifying process by searching the similar datathat is similar to the input data as indicated in the above (b), the tagidentifying section 105 performs a similarity retrieval of data that theuser or a large-scale user group (e.g. a user group in a network servicesuch as an SNS) has tagged in the past by using the input datacorresponding to the process target area. Then, the tag identifyingsection 105 handles the tag associated with the data determined as beingsimilar to the input data as the tag corresponding to the input data.For example, in a case of performing the identifying process by thesimilar data search for the image in which a food is photographed (foodimage), the tag identifying section 105 performs a similaritydetermining process of the image data corresponding to the processtarget area and an image group that had been tagged by the user or thelarge-scale user group in the past. Then, if an image determined asbeing similar to the input image data exists, the tag identifyingsection 105 handles the food tag associated with the image determined asbeing similar as the food tag corresponding to the input image data.

Further, if there is a plurality of process target areas in image data,it is possible for the tag identifying section 105 to use theidentifying process based on the association history of the tags in thetree structure as indicated in the above (c). This method is a methodfor identifying a tag of an unidentified process target area that isused in a case where a tag of another process target area existing inthe same input data is already identified, based on a co-occurrencerelationship of the tag associated with the already-identified processtarget area and the tagging that the user or the large-scale user grouphad done in the past. In this method, e.g. when a plurality of food isincluded in input image data and an identification result of the tag forfood other than the food at focus is already obtained, food with a highchance of being selected is determined as an identification result fromcombinations of the tagging result of the food to which theidentification result has already been obtained and the food tagged bythe user or the large-scale user group in the past. Specifically, whenthe other food included in an image has already been tagged as “misosoup”, and if the co-occurrence relationship that “the user or thelarge-scale users in many cases have a meal with a combination of “misosoup” and “rice”” exists, then the tag identifying section 105identifies the food included in the unidentified process target area as“rice”.

By using the methods as explained above, the tag identifying section 105can identify tags independently for the process target areas in theinput data.

Note that, the tag identifying section 105 may use one of the methods asin the above (a) to (c) independently, or may use them in combinations.Further, in using a plurality of methods in combinations, the tagidentifying section 105 may integrally determine the identificationresults by the respective identifying methods, and may select a tagwhich appears more certain as the identification result.

Further, when a tag corresponding to the input data is identified, thetag identifying section 105 also outputs an evaluation value of theidentification result of the tag (a score of the identification result,similarity score showing similarity, coincidence to the history, and thelike). This evaluation value indicates how certain the identificationresult of the tag is (likelihood thereof), and it can be determined e.g.that as the value thereof is greater, the tag having that value has ahigher chance of corresponding to the input data. A calculation methodof the evaluation value of the identification result of the tag is notparticularly limited, and any known technique may be used.

The tag identifying section 105 specifies a corresponding tag for eachof the process target areas included in the input data according to theabove, and when an evaluation value related to the specified tag iscalculated, creates information (tag identification information)including a specification result of the tag (input-data-correspondingtag) and the evaluation value (tag identification information). Then,the tag identifying section 105 outputs the created tag identificationinformation to the tag candidate extracting section 107 to be describedlater. Note that, the tag identifying section 105 may store the createdtag identification information as history in the storing section 115 tobe described later.

The tag candidate extracting section 107 is implemented e.g. by a CPU,ROM, RAM, and the like. The tag candidate extracting section 107extracts, for each granularity that is to be the reference of theinclusion relation of the events or concepts, a tag that may correspondto the input data from among the plurality of tags respectivelyassociated with the tree structure, based on the input data including atleast one of a character, image or sound. Further, the tag candidateextracting section 107 may further extract a tag in a directly upperhierarchical level including a tag belonging to a same hierarchicallevel as extracted, in accordance with a circumstance of extraction ofthe tag belonging to the same hierarchical level in the tree structure.

As an example of the circumstance of the extraction of the tag belongingto the same hierarchical level in the tree structure, e.g. conditions asbelow may be exemplified. For example, in cases where one of thefollowing conditions is satisfied, the tag candidate extracting section107 may further extract a parent node of a relevant child node group asa tag candidate. Note that, the following conditions are merely anexample, and conditions for determining whether to extract the tag inthe directly upper hierarchical level or not are not limited to thefollowing conditions.

-   -   Whether a tag candidate extracted from the child nodes exists in        excess of a predetermined threshold or not    -   Whether a tag candidate extracted from the child nodes exists in        excess of a predetermined ratio or not    -   Whether an average in identification scores of the child nodes        is at or more than a predetermined threshold or not

Here, as explained earlier, the tree structure that the tag candidateextracting section 107 will use in extracting the tag candidateindicates the inclusion relation of the events or concepts. As shownschematically e.g. in FIG. 3, the tree structure that the tag candidateextracting section 107 will use may indicate the events or concepts inthe form of the tree structure, or may be a database or lookup table asshown in FIG. 6 with the food tags as the example in which the inclusionrelation of matters such as the names of the food and classes.

In the example of the tree structure related to food tags shown in FIG.6, the names of the food are described as small classes (in other words,leaf nodes in the tree structure as shown in FIG. 3), names of middleclasses that are the classification distinction defining the food of thecorresponding leaf nodes in broader terms as being the upperhierarchical level of the small classes, and names of large classes thatare the classification distinction further defining the middle classesin broader terms are described.

In other words, the tree structure shown in FIG. 6 has the food tag “alltypes of food” as a root node, various nodes (food tags) relevant to the“large classes” as the nodes belonging to a first hierarchical level,various nodes relevant to the “middle classes” as the nodes belonging toa second hierarchical level, and various leaf nodes relevant to the“small classes” as the nodes belonging to a third hierarchical level.Further, in the tree structure shown in FIG. 6, nodes belonging to thesame hierarchical level can be said as being a cluster havinggranularities similar to one another. The respective leaf nodesbelonging to the third hierarchical level are the food tags used infinal tagging of the input data.

Note that, a method of classifying the tree structure related to thefood tags as shown in FIG. 6 is merely an example, and the granularitiesthat are to be the references for the inclusion relation fordistinguishing the middle classes and the large classes may bereferences as follows.

-   -   Depending on ingredients (grain, meat, fish, . . . , etc.)    -   Depending on cooking methods (baked, steamed, stewed, . . . ,        etc.)    -   Depending on regions (Japanese, Chinese, Italian, French, . . .        , etc.)

Hereinbelow, the tag candidate extracting process by the tag candidateextracting section 107 will be specifically explained with reference toFIG. 7A to FIG. 7C with the tag candidate extracting process related tofood tags as examples. Below, a portion of the tree structure related tothe food tags in which the small class nodes in which discriminatorsexist and the small class nodes in which no discriminators exist aremixedly present as shown in FIG. 7A will be exemplified.

A case will be considered in which image data having photographed foodbelonging to noodles is notified as the input data to the tag candidateextracting section 107 that performs the extraction of tag candidatesusing a portion of the tree structure as shown in FIG. 7A. In this case,as shown in FIG. 7B, the tag candidate extracting section 107 firstlyreferences the identification score included in the identificationresult outputted from the tag identifying section 105, and extracts thetag candidates according to the threshold determination of theidentification score.

The tag candidate extracting section 107 references the identificationscore outputted from the tag identifying section 105, and determineswhether the identification score is at or more than an extract thresholdfor each of the discriminators or not. In the example shown in FIG. 7B,the tag candidate extracting section 107 is assumed as having extractedfour food tags of: “tomato sauce”, “miso ramen noodle”, “ramen noodlewith roasted pork”, and “ramen noodle with vegetables”.

Next, as shown in FIG. 7C, the tag candidate extracting section 107performs extraction of a tag candidate according to thresholddetermination of a number of selected child nodes. Here, as the extractthreshold by the number of selected child nodes, a condition thatwhether an extracted number of directly lower small classes is 2 or moreor not is set as the condition for extracting the hierarchical levelrelevant to the middle class, and a condition that whether an extractednumber of directly lower middle classes is 1 or more or not is set asthe condition for extracting the hierarchical level relevant to thelarge class.

As is apparent from the extraction result in step 1 shown in FIG. 7B,three child nodes (small classes) belonging to the middle class “ramennoodles” are extracted. This extracted number satisfies the conditionfor extracting the tag relevant to the middle class. Accordingly, asshown in FIG. 7C, the tag candidate extracting section 107 furtherextracts the middle class “ramen noodles” as a tag candidate based onthe circumstance of extraction of the small class.

Further, with the middle class “ramen noodles” being extracted, thecondition for extracting the tag relevant to the large class issatisfied. Accordingly, the tag candidate extracting section 107 furtherextracts the large class “noodles” as a tag candidate based on thecircumstance of extraction of the middle class.

From the aforementioned processes, the tag candidate extracting section107 extracts for the respective granularities the six types of foodtags, namely: the large class “noodles”, middle class “ramen noodles”,and small classes “tomato sauce”, “miso ramen noodle”, “ramen noodlewith roasted pork”, and “ramen noodle with vegetables”, as the tagcandidates that may be relevant to the input data.

Further, since the tag candidates extracted by the tag candidateextracting section 107 according to the above processes are selected foreach of the granularities, they construct a new tree structureconfigured of tags with high chances of being selected by the userinstead of the overall tree structure that is prepared in advance.

Further, the tag candidate extracting section 107 may interpolate theextracted tags (tag candidates) by using tags related to the extractedtags and having high chances of being selected as the tag correspondingto the input data. There may be examples as follows of the tags havingthe high chances of being selected as the tag corresponding to the inputdata.

-   -   Tags popular to the user (in the example of the food tags,        popular menu, standard menu and the like)    -   Tags among the child nodes with a large number of tagging in the        past by the user or the large-scale user group    -   Identification results based on tendencies of histories of the        user or the large-scale user group

By further extracting such tags, it becomes possible to extract tags byfiltering even for a hierarchical level in which tags in the treestructure does not exist, and the user's convenience can further beimproved.

For example, as shown in FIG. 8, a case will be considered in which “soysauce ramen noodle”, “pork bone broth ramen noodle”, and “instant ramennoodle” exist as the small classes belonging to the middle class “ramennoodles”, and “plain udon”, “udon with deep-fried tofu”, and “udon withtempura crunches” exist as the small classes belonging to a middle class“udon noodles” as the menu with a large number of past selections by theuser or the large-scale user group. In this occasion, in addition to thetag candidates extracted in FIG. 7C, the tag candidate extractingsection 107 also extracts tags as shown in FIG. 8 as the tag candidates.

After having extracted the tag candidates that may be relevant to theinput data as aforementioned, the tag candidate extracting section 107creates tag candidate information indicating the extraction results ofthe tag candidates, and outputs the same to the tag candidateinformation outputting section 109 to be described later. This tagcandidate information may include not only the information indicatingthe extracted tag candidates, but also various metadata regarding theextracted tag candidates such as the evaluation values and the like ofthe extracted tag candidates. Further, the tag candidate extractingsection 107 may associate the created tag candidate information with theinput data corresponding to the tag candidate information, and store thesame in the storing section 115 and the like as a history.

According to the above, the functions of the tag candidate extractingsection 107 have been explained specifically with reference to FIG. 6 toFIG. 8.

Hereinbelow, by returning to FIG. 5, the tag candidate informationoutputting section 109 will be explained.

The tag candidate information outputting section 109 is implemented e.g.by a CPU, ROM, RAM, communicating device, and the like. The tagcandidate information outputting section 109 outputs the tag candidateinformation that is information created by the tag candidate extractingsection 107 and related to the extraction results of the tag candidates,to the information processing device 20 that had designated the inputdata corresponding to the tag candidate information. Due to this, theextraction results of the tag candidates by the tag candidate extractingsection 107 are notified to the information processing device 20 thathad designated the input data, and it becomes possible for theinformation processing device 20 to present information regarding theextracted tag candidates to the user.

The user operation information acquiring section 111 is implemented e.g.by a CPU, ROM, RAM, communicating device, and the like. The useroperation information acquiring section 111 acquires user operationinformation indicating a result of tag selection outputted from theinformation processing device 20 with respect to the input data (whichtag among the tag candidates has been selected as the tag correspondingto the input data by the user operation). When the user operationinformation indicating the result of tag selection by the user isacquired, the user operation information acquiring section 111 outputsthe acquired user operation information to the tag allotting section 113to be described later.

The tag allotting section 113 is implemented e.g. by a CPU, ROM, RAM,and the like. The tag allotting section 113 specifies the tag selectedby the user from among the tag candidates extracted by the tag candidateextracting section 107 based on the user operation informationindicating the result of tag selection by the user outputted from theuser operation information acquiring section 111, and allots the tagselected by the user as the tag corresponding to the input data.Further, in a case where a tag other than the tag candidates extractedby the tag candidate extracting section 107 is selected by the useroperation, the tag selected by the user is allotted as the tagcorresponding to the input data. Due to this, the tag corresponding tothe input data is specified.

The storing section 115 is implemented e.g. by a RAM, a storage device,and the like. The storing section 115 stores various discriminators usedby the tag identifying section 105, various tree structures used by thetag candidate extracting section 107, and the like. Further, the storingsection 115 may store various programs, various parameters that had tobe stored upon performing some process, progresses of the process by theinformation processing server 10 according to the present embodiment, orvarious databases and the like, as appropriate. Further, in the storingsection 115, the input data including at least one of a character, imageor sound may be stored.

This storing section 115 can be freely accessed by respective processingsections such as the data acquiring section 101, process target areadetecting section 103, tag identifying section 105, tag candidateextracting section 107, tag candidate information outputting section109, user operation information acquiring section 111, tag allottingsection 113, and the like, and data can be written and read thereby.

According to the above, an example of the functions of the informationprocessing server 10 according to the present embodiment has beenpresented. The respective constituent features as above may beconfigured by multi-purposed members and circuits, or may be configuredof hardware dedicated to the functions of the respective constituentfeatures. Further, all of the functions of the respective constituentfeatures may be performed by a CPU and the like. Accordingly, it ispossible to modify the configuration to be used as appropriate dependingon the technical levels at which the present embodiment is to be putinto practice.

Note that, it is possible to create a computer program for implementingthe respective functions of the information processing server accordingto the present embodiment as above, and install the same in a personalcomputer and the like. Further, a computer readable recording medium inwhich such a computer program is stored may be provided. The recordingmedium may e.g. be a magnetic disc, optical disc, magneto-optical disc,flash memory, and the like. Further, the computer program may bedelivered through e.g. a network without using the recording medium.

<Configuration of Information Processing Device>

Next, a configuration of the information processing device 20 accordingto the present embodiment will be explained in detail with reference toFIG. 9 to FIG. 15. FIG. 9 is a block diagram showing an example of theconfiguration of the information processing device 20 according to thepresent embodiment. FIG. 10 to FIG. 15 are explanatory diagrams showingexamples of a display controlling process according to the presentembodiment.

As shown in FIG. 9, the information processing device 20 according tothe present embodiment primarily includes a user operation informationacquiring section 201, user operation information outputting section203, tag candidate information acquiring section 205, display controller207, and storing section 209.

The user operation information acquiring section 201 is implemented e.g.by a CPU, ROM, RAM, input device, and the like. The user operationinformation acquiring section 201 specifies an operation (useroperation) that the user had performed to an input device provided inthe information processing device 20 such as a mouse, keyboard, touchpanel, gesture input device, sight input device, and the like, andcreates user operation information regarding the user operation.Thereafter, the user operation information acquiring section 201 outputsthe created user operation information to the user operation informationoutputting section 203, tag candidate information acquiring section 205,display controller 207, and the like to be described later. Due to this,it becomes possible to grasp what kind of operation the user hadperformed on the information processing device 20, and it becomespossible to provide the function corresponding to the user operation tothe user.

The user operation information outputting section 203 is implementede.g. by a CPU, ROM, RAM, communicating device, and the like. The useroperation information outputting section 203 outputs the user operationinformation to the information processing server 10. The user operationinformation is information to be used in various processes in theinformation processing server 10 among the user operation informationoutputted from the user operation information acquiring section 201,such as the user operation information related to designation of theinput data, and the user operation information indicating the tagselected by the user from among the tag candidates. Note that, the useroperation information that the user operation information outputtingsection 203 outputs to the information processing server 10 are notlimited to the above examples.

The tag candidate information acquiring section 205 is implemented e.g.by a CPU, ROM, RAM, communicating device, and the like. The tagcandidate information acquiring section 205 acquires tag candidateinformation describing the extraction results obtained by extractingtags that may correspond to the input data for each granularity (i.e.tag candidates) from among the plurality of tags outputted from theinformation processing server 10 and associated respectively with thetree structure. Upon acquiring the tag candidate information from theinformation processing server 10, the tag candidate informationacquiring section 205 outputs the acquired tag candidate information tothe display controller 207 to be described later. Further, the tagcandidate information acquiring section 205 may associate the acquiredtag candidate information with time information regarding the time atwhich the tag candidate information was acquired, and store the same inthe storing section 209 and the like as a history.

The display controller 207 is implemented e.g. by a CPU, ROM, RAM,output device, communicating device, and the like. The displaycontroller 207 acquires data stored in the storing section 209 and thelike and corresponding to contents to be displayed on a display screen,and displays the same on the display screen. Further, if a signalindicating a movement of a position selecting object such as a pointeris transmitted from the input device provided in the informationprocessing device 20 such as the mouse, keyboard, touch panel, gestureinput device, sight input device, and the like, the display controller207 displays the movement of the position selecting object on thedisplay screen in accordance with the transmitted signal.

Further, in a case where a display of the tag candidates extracted fromthe information processing server 10 is requested by the tag candidateinformation acquiring section 205, the display controller 207 changesthe displayed contents that are displayed on the display screen based onthe request from the tag candidate information acquiring section 205.More specifically, when the tag candidate information describing theextraction results obtained by extracting tags that may correspond tothe input data for each of the granularities from among the plurality oftags associated respectively with the tree structure is acquired fromthe tag candidate information acquiring section 205, the displaycontroller 207 performs a control for displaying the tag candidates onthe display screen for each granularity based on this tag candidateinformation.

Here, as for the display control performed by the display controller 207based on the tag candidate information notified from the tag candidateinformation acquiring section 205 will be explained below with anindication of a specific example.

The storing section 209 is implemented e.g. by a RAM, storage device,and the like. In the storing section 209, object data to be displayed onthe display screen are stored. The object data referred to hereininclude voluntary parts configuring a graphical user interface (GUI)such as icons, buttons, thumbnails and the like. Further, the storingsection 209 may store various programs including applications to beexecuted by the information processing device 20 according to thepresent embodiment, various parameters that had to be stored uponperforming some process, progresses of the process, or various databasesand the like, as appropriate. Further, in the storing section 209,various data including at least one of a character, image or sound andthat may be handled as the input data may be stored.

This storing section 209 can be freely accessed by respective processingsections such as the user operation information acquiring section 201,user operation information outputting section 203, tag candidateinformation acquiring section 205, display controller 207 and the like,and data can be written and read thereby.

[Example of Graphical User Interface]

Next, an example of the graphical user interface (GUI) provided to theuser by the information processing device 20 will be explainedspecifically with reference to FIG. 10 to FIG. 15. Note that, in theexample shown below, an example of the GUI that the informationprocessing device 20 according to the present embodiment can provide tothe user is exemplified; and the GUI that the information processingdevice 20 according to the present embodiment provides to the user isnot limited to the following example.

Note that, in the following explanation, it is assumed that the tagcandidates shown in FIG. 8 are extracted by the information processingserver 10 based on the input data designated by the informationprocessing device 20, and the tag candidate information regarding theextracted tag candidates is outputted from the information processingserver 10.

In the case where the display screen control for providing the user withthe tag candidates is requested from the tag candidate informationacquiring section 205, the display controller 207 provided in theinformation processing device 20 according to the present embodimentcreates a thumbnail display area in which the thumbnail of the inputdata is to be displayed and a tag candidate display area in which thecontents of the tag candidate information are to be displayed on thedisplay screen. Thereafter, the display controller 207 displays thethumbnail corresponding to the input data in the thumbnail display area,and displays the tag candidates for each of the granularities describedin the tag candidate information in the tag candidate display area.

Note that, how the thumbnail display area and the tag candidate displayarea are arranged in the display screen is not particularly limited, anda voluntary layout may be employed.

For example, in the GUI shown in FIG. 10, the display controller 207displays the tag candidates described in the tag candidate informationin the tag candidate display area in a form of the tree structure inaccordance with the granularity. By indicating the tree structure asabove, the user can grasp the tags that may be relevant to the inputdata for each of the granularities.

As in the tag e.g. of “pastas” in FIG. 11, the display controller 207may auxiliarily provide tags that were not extracted as the tagcandidates to the user to improve the user's convenience. Further, thedisplay controller 207 may change the display format of the tags basedon the evaluation values included in the tag candidate information. Thatis, the display controller 207 may sort the tag candidates in an orderof high evaluation values and display the same in the display screen;and as exemplified in FIG. 11, a display format of tag names (e.g. fonttype, color, size, thickness, etc.) may be changed in accordance withthe evaluation values.

Further, as shown e.g. in FIG. 12, the display controller 207 maydisplay a list or a table of the tag candidates described in the tagcandidate information in accordance with the granularity. Further, thedisplay controller 207 may display an object for displaying undisplayedtags also on the display screen, and may perform a display screencontrol to display the undisplayed tags in the tag candidate displayarea when this object is selected by the user.

Further, as shown in FIG. 13, the display controller 207 can provide theuser with a searching function for tag candidates by concurrentlydisplaying a search word input area on the display screen. As shown inthe left side of FIG. 13, in an initial state in which no search word isinput, the display controller 207 displays the tag candidates describedin the tag candidate information in the tag candidate display area.Thereafter, when a letter string is inputted by a user operation in thesearch word input area, as shown in the right side of FIG. 13, the tagcandidates may be narrowed down based on the search word, and the resultthereof may be displayed. Further, at this occasion, tags that arerelevant to the search word but are not included in the tag candidatemay concurrently be displayed in the tag candidate display area.

Further, the GUI provided by the display controller 207 is not limitedto the examples shown in FIG. 10 to FIG. 13; for example, it may bemodified appropriately in accordance with the size, etc. of the displayscreen. For example, in the GUI shown in FIG. 14, the tag candidates areshown in a list and the thumbnail images of the corresponding tagcandidates are arranged in the vicinity of the names of the respectivetag candidates, so as to make the respective display areas more compact.Further, in the GUI shown in FIG. 14, classification selecting objects(e.g. icons, etc.) for selecting the granularity are displayed on thedisplay screen so as to improve the user operability.

Note that, in the display controller 207, e.g. as shown in FIG. 15, auser interface for newly adding tags not described in the tag candidatesin accordance with a user operation may be provided. Due to this, if adesired tag does not exist in the tag candidate display area, the usercan newly create a tag and add the same. For example, in the exampleshown in FIG. 15, a small class “Szechuan sesame and chili ramen noodle”belonging to the middle class “ramen noodles” is added in accordancewith the user operation.

Accordingly, the examples of the graphical user interface (GUI)presented to the user by the information processing device 20 have beenexplained specifically with reference to FIG. 10 to FIG. 15.

As described above, the examples of the functions of the informationprocessing device 20 according to the present embodiment have beenpresented. The respective constituent features as above may beconfigured by multi-purposed members and circuits, or may be configuredof hardware dedicated to the functions of the respective constituentfeatures. Further, all of the functions of the respective constituentfeatures may be performed by a CPU and the like. Accordingly, it ispossible to modify the configuration to be used as appropriate dependingon the technical levels at which the present embodiment is to be putinto practice.

Note that, it is possible to create a computer program for implementingthe respective functions of the information processing device accordingto the present embodiment as above, and install the same in a personalcomputer and the like. Further, a computer readable recording medium inwhich such a computer program is stored may be provided. The recordingmedium may e.g. be a magnetic disc, optical disc, magneto-optical disc,flash memory, and the like. Further, the computer program may bedelivered through e.g. a network without using the recording medium.

<As to Another Example Regarding Input Data>

Note that, in the above explanation, the example of performing thetagging process of food with the image data in which food isphotographed as the input data by the information processing server 10and the information processing device 20 according to the presentembodiment had been exemplified, however, the input data according tothe present embodiment is not limited to the above example.

Tag candidates may be extracted with respect to image data in which anobject other than food (e.g. a person's face, etc.) exists. For example,in the example shown in FIG. 16, an example in which the informationprocessing server 10 detects the face included in the input image databy a known face recognition technique, and performs a known faceidentifying process based on face identification attributes such as aperson's sex, age, race, presence/absence of glasses and the like, andextracts tag candidates of person's names is exemplified.

Further, the input data that the information processing server 10 andthe information processing device 20 according to the present embodimenthandle as the process target is not limited to data including an image,but such may be data including sound (audio data) e.g. as shown in FIG.17. If audio data is designated as the input data, the informationprocessing server 10 e.g. extracts a process target area from a speechwaveform, and thereafter, the extraction of the tag candidates isperformed based on known sound recognition technique and voiceprintrecognition technique as well as a tree structure that ispredeterminedly constructed. By performing such processes, as shown inFIG. 17, the information processing server 10 can extract the tagcandidates regarding the person's names based on the audio data, and thetag candidates regarding environmental sounds.

Further, the input data that the information processing server 10 andthe information processing device 20 according to the present embodimenthandle as the process target is not limited to data including an image,but may e.g. be text data as shown in FIG. 18. If text data isdesignated as the input data, the information processing server 10 e.g.extracts tag candidates regarding types of the text and tag candidatesregarding emotions which the text expresses, as shown in FIG. 18, byusing e.g. a known language recognition technique and pattern matchingtechnique, as well as a tree structure that is predeterminedlyconstructed.

Further, the input data that the information processing server 10 andthe information processing device 20 according to the present embodimenthandle as the process target may be mixed data of an image, sound, andcharacters.

<As to Information Extracting Method and Information Processing Method>

Next, as to an information extracting method and information processingmethod performed by the information processing server 10 and theinformation processing device 20 according to the present embodiment,flows thereof will briefly be explained with reference to FIG. 19. FIG.19 is a flow chart showing an example of the flows of the informationextracting method and information processing method according to thepresent embodiment.

When the user operation information for designating the input data isacquired, the user operation information acquiring section 201 of theinformation processing device 20 according to the present embodimentoutputs the acquired user operation information to the user operationinformation outputting section 203. Thereafter, the user operationinformation outputting section 203 outputs the user operationinformation that designates the input data to the information processingserver 10. Due to this, the input data that the information processingserver 10 handles as the process target is designated (step S101).

When the user operation information that designates the input data asoutputted from the information processing device 20 is acquired, theinformation processing server 10 notifies information regarding locationof the designated input data to the data acquiring section 101. The dataacquiring section 101 acquires the relevant input data based on thelocation information of the input data as notified (step S103), andoutputs the acquired input data to the process target area detectingsection 103.

Thereafter, the process target area detecting section 103 detects theprocess target area from among the input data as outputted from the dataacquiring section 101 by using a known technique (step S105), andoutputs the information indicating the input data and the detectionresult to the tag identifying section 105.

The tag identifying section 105 identifies tags corresponding to theinput data by using the data relevant to the process target data amongthe input data (step S107), and creates information regarding theidentification results of the tags. Thereafter, the tag identifyingsection 105 outputs the created information regarding the identificationresults of the tags to the tag candidate extracting section 107.

The tag candidate extracting section 107 extracts the tag candidates byusing the predeterminedly created tree structure and the informationregarding the identification results of the tags outputted from the tagidentifying section 105, according to the methods as explained earlier(step S109). Then, the tag candidate extracting section 107 creates thetag candidate information regarding the extracted tag candidates, andoutputs the same to the tag candidate information outputting section109.

The tag candidate information outputting section 109 outputs the tagcandidate information as notified by the tag candidate extractingsection 107 to the information processing device 20 (step S111). Due tothis, information regarding the tag candidates that may be relevant tothe input data designated by the information processing device 20 areprovided to the information processing device 20.

When the tag candidate information outputted from the informationprocessing server 10 is acquired, the tag candidate informationacquiring section 205 of the information processing device 20 outputsthe acquired tag candidate information to the display controller 207.The display controller 207 performs the display screen control forproviding the user with the tag candidates extracted by the informationprocessing server 10 based on the tag candidate information outputtedfrom the tag candidate information acquiring section 205 (step S113).

When the tag candidate corresponding to the input data is selected bythe user operation, the user operation information acquiring section 201creates the user operation information indicating the selection resultby the user and outputs the same to the user operation informationoutputting section 203. The user operation information outputtingsection 203 outputs the user operation information indicating theselection result by the user regarding the tag to the informationprocessing server 10 (step S115).

When the user operation information indicating the selection result bythe user regarding the tag as outputted from the information processingdevice 20 is acquired, the user operation information acquiring section111 of the information processing server 10 outputs the acquired useroperation information to the tag allotting section 113. The tagallotting section 113 allots the tag selected by the user as the tagcorresponding to the input data based on the user operation informationoutputted from the user operation information acquiring section 111(step S117). Due to this, the tag corresponding to the input data isfinalized, and location relationships in the tree structure isfinalized.

As described above, the flows of the information extracting method andthe information processing method performed by the informationprocessing server 10 and the information processing device 20 accordingto the present embodiment have been briefly explained with reference toFIG. 19.

<Variant>

The functions of the information processing server shown in FIG. 5 andthe functions of the information processing device 20 shown in FIG. 9may be implemented in either hardware so long as the both hardware cansend and receive information to and from one another through a network.Further, a process to be performed by a particular processing sectionmay be implemented by one hardware, or may be implemented by adistributed processing by a plurality of hardware.

In the variant shown e.g. in FIG. 20, an example in which the functionsof the information processing server 10 shown in FIG. 5 and thefunctions of the information processing device 20 shown in FIG. 9 areimplemented in one device is shown.

The information processing device 30 shown in FIG. 20 according to thevariant primarily includes a user operation information acquiringsection 301, data acquiring section 303, process target area detectingsection 305, tag identifying section 307, tag candidate extractingsection 309, display controller 311, data allotting section 313, andstoring section 315.

Here, the user operation information acquiring section 301 has similarfunctions as the user operation information acquiring section 201 shownin FIG. 9 except for that it outputs the acquired user operationinformation to the data acquiring section 303, display controller 311,and tag allotting section 313, and similar effects can be achieved.Further, the display controller 311 has similar functions as the displaycontroller 207 shown in FIG. 9 except for that it performs the displaycontrol based on the tag candidate information outputted from the tagcandidate extracting section 309, and similar effects can be achieved.Accordingly, hereinbelow detailed explanations regarding theseprocessing sections will be omitted.

Further, the data acquiring section 303, process target area detectingsection 305, tag identifying section 307, tag candidate extractingsection 309, tag allotting section 313, and storing section 315 havesimilar functions as the data acquiring section 101, process target areadetecting section 103, tag identifying section 105, tag candidateextracting section 107, tag allotting section 11, and storing section115 respectively shown in FIG. 5, and similar effects can be achieved.Accordingly, hereinbelow detailed explanations regarding theseprocessing sections will be omitted.

As described above, the variant of the information processing server 10and information processing device 20 according to the embodiment 1 wasbriefly be explained with reference to FIG. 20.

(Hardware Configuration)

Now referring to FIG. 21, hardware configuration of the informationprocessing server 10 according to the embodiment of the presentdisclosure will be described in detail. FIG. 21 is a block diagram forexplaining the hardware configuration of the information processingserver 10 according to the embodiment of the present disclosure.

The information processing server 10 includes mainly a CPU 901, a ROM903 and a RAM 905. The information processing server 10 further includesa host bus 907, a bridge 909, an external bus 911, an interface 913, aninput device 915, an output device 917, a storage device 919, a drive921, a connection port 923 and a communicating device 925.

The CPU 901 functions as an arithmetic processing unit and a controldevice to control entire or a part of operation in the informationprocessing server 10 in accordance with various kinds of programsrecorded in the ROM 903, RAM 905, storage device 919 or removable recordmedium 927. The ROM 903 stores programs, operation parameters and thelike used by the CPU 901. The RAM 905 temporarily stores programs usedby the CPU 901, and parameters which are appropriately changed duringexecuting the programs. These are connected to each other through a hostbus 907 including an internal bus such as CPU bus.

The host bus 907 is connected to an external bus 911 such as PCI(peripheral component interconnect/interface) bus via a bridge 909.

The input device 915 is an operation device for allowing a user tooperate thereon including, for example, a mouse, a keyboard, a touchpanel, a button, a switch, a lever and the like. The input device 915may be, for example, a remote control device (so-called, remote) whichuses infrared light or other radio wave, or an external connectiondevice 929 such as a mobile phone, a PDA or the like corresponding tothe operation of the information processing server 10. The input device915 further includes, for example, an input control circuit whichgenerates an input signal based on information input by a user andoutputs the same to the CPU 901 using the above-described operationdevice. By operating the input device 915, a user of the informationprocessing server 10 is able to input various kinds of data to give aninstruction of a processing operation to the information processingserver 10.

The output device 917 includes a device which is capable of providingobtained information to a user in a visual or auditory manner. As suchdevice, display devices including a CRT display device, a liquid crystaldisplay device, a plasma display device, an EL display device and a lampand the like; audio output devices such as speaker, head phone and thelike; a printer unit; a mobile phone; a facsimile and the like areavailable. The output device 917 outputs, for example, a result obtainedby various kinds of processing made by the information processing server10. In particular, the display device displays the result of variouskinds of processing made by the information processing server 10 in aform of text or an image. On other hand, an audio output device convertsaudio signals of reproduced voice data or acoustic data into analogsignals and outputs the same.

The storage device 919 is an example of a storage device configured forstoring data of the information processing server 10. The storage device919 may be, for example, magnetic memory devices such as a HDD (harddisk drive), a semiconductor memory device, an optical memory device oran optical magnetic memory device. The storage device 919 stores aprogram executed by the CPU 901, various kinds of data, and variouskinds of data obtained from the outside.

The drive 921 is a reader/writer for record medium, which is included inthe information processing server 10 or externally provided thereto. Thedrive 921 reads information recorded in a magnetic disk, an opticaldisk, a magnetic optical disk, or a removable record medium 927 such assemiconductor memory or the like mounted thereon, and outputs the sameto the RAM 905. The drive 921 can also write a record on a magneticdisk, an optical disk, a magnetic optical disk mounted thereon, or aremovable record medium 927 such as semiconductor memory or the like.The removable record medium 927 may be, for example, a DVD media, aHD-DVD media, a Blu-ray media or the like. The removable record medium927 may be a CompactFlash (registered mark), a flash memory, or an SDmemory card (secure digital memory card) or the like. The removablerecord medium 927 may be, for example, an IC card (integrated circuitcard) mounted with non-contact IC chip or an electronic device.

The connection port 923 is a port for directly connecting a device tothe information processing server 10. As an example of the connectionport 923, a USB (universal serial bus) port, an IEEE 1394 port, an SCSI(small computer system interface) port and the like are available. Asanother example of the connection port 923, an RS-232C port, an opticalaudio terminal, an HDMI (high-definition multimedia interface) port andthe like are available. By connecting the external connection device 929to the connection port 923, the information processing server 10 obtainsvarious kinds of data directly from the external connection device 929and provides various kinds of data to the external connection device929.

The communicating device 925 is a communication interface including, forexample, a communication device or the like for connecting tocommunication network 931. The communicating device 925 may be, forexample, a wired or wireless LAN (local area network), Bluetooth(registered mark) or a communication card for WUSB (Wireless USB) or thelike. The communicating device 925 may be a router for opticalcommunication, a router for ADSL (asymmetric digital subscriber line) ora modem for various kinds of communication. The communicating device 925is capable of transmitting and receiving signals via, for example,Internet or other communication device in accordance with apredetermined protocol like, for example, TCP/IP. The communicationnetwork 931 connected to the communicating device 925 may include anetwork or the like connected in a wired or wireless manner such as forexample, Internet, a home LAN, an infrared communication, aradiofrequency communication or a satellite communication.

A example of hardware configurations capable of achieving the functionsof the information processing server 10 according to the embodiment ofthe present disclosure has been described above. The above-describedcomponent elements may include a general purpose unit or circuit, orhardware each specialized to the functions of the component elements maybe included. Therefore, the applied configuration may be appropriatelychanged in accordance with the technical art at the point when theembodiment is implemented.

The above-described hardware configuration of the information processingdevice 20 according to the embodiment of the present disclosure has thesame configuration as the hardware configuration of the informationprocessing server 10 according to the embodiment of the presentdisclosure. Therefore, detailed description is omitted here.

CONCLUSION

As described, in the embodiments of the present disclosure, the tagcandidates that may correspond to the input data are extracted accordingto the methods as described above by using the predeterminedly createdtree structure in an order with priority to tags with higher chances ofbeing selected, and the extracted tag candidates are presented in amanner with which the user can more easily perform tagging. Due to this,according to the embodiments of the present disclosure, an amount ofuser's input procedures in the tagging process can be reduced, and theuser's convenience can be improved.

Although the preferred embodiments of the present disclosure have beendescribed in detail with reference to the appended drawings, the presentdisclosure is not limited thereto. It is obvious to those skilled in theart that various modifications or variations are possible insofar asthey are within the technical scope of the appended claims or theequivalents thereof. It should be understood that such modifications orvariations are also within the technical scope of the presentdisclosure.

Particular embodiments of the present technology include the following.

(1) An information processing system including: a processor fordetermining one or more candidate tags based on input data, thecandidate tags being included within a hierarchical structure; and adisplay for displaying the candidate tags in a manner indicative of thecandidate tags' positions in the hierarchical structure.

(2) The system according to (1), wherein the processor and display areimplemented in a single device.

(3) The system according to (1), wherein the system comprises at leastone information processing server and at least one informationprocessing device, the processor being included in the informationprocessing server, and the display being included in the informationprocessing device.

(4) The system according to (1), wherein an evaluation value isgenerated for each candidate tag, the evaluation value being indicativeof the candidate tag's correspondence with the input data.

(5) The system according to (4), wherein the candidate tags aredisplayed in order of descending evaluation values.

(6) The system according to (4), wherein the candidate tags aredisplayed in a manner that emphasizes selected candidate tags accordingto the evaluation values.

(7) The system according to (1), wherein the display simultaneouslydisplays the candidate tags and the input data.

(8) The system according to (1), wherein the candidate tags aredisplayed in a tree format.

(9) The system according to (1), wherein the display simultaneouslydisplays the candidate tags, the input data, and a candidate tag searchbox.

(10) The system according to (9), wherein as text is entered in thesearch box fewer candidate tags are displayed.

(11) The system according to (1), wherein the processor determines oneor more candidate tags by determining a process target area of the inputdata and determining one or more candidate tags based on the processtarget area.

(12) The system according to (11), wherein the process target area isdetermined according to a manual input of a user.

(13) The system according to (11), wherein the process target area isdetermined automatically.

(14) The system according to (11), wherein the input data is image dataand the processor performs an image recognition process to determine theprocess target area.

(15) The system according to (11), wherein the input data is text dataand the processor performs a language recognition process to determinethe process target area.

(16) The system according to (11), wherein the input data is sound dataand the processor performs a sound recognition process to determine theprocess target area.

(17) The system according to (1), wherein, for each displayed candidatetag, a corresponding thumbnail image is displayed in the vicinity of thecandidate tag.

(18) The system according to (1), wherein an object for initiatingdisplay of undisplayed tags is displayed with the displayed candidatetags.

(19) An information processing method including: determining one or morecandidate tags based on input data, the candidate tags being includedwithin a hierarchical structure; and displaying the candidate tags in amanner indicative of the candidate tags' positions in the hierarchicalstructure.

(20) A non-transitory computer-readable medium having stored thereon acomputer-readable program for implementing an information processingmethod, the method including: determining one or more candidate tagsbased on input data, the candidate tags being included within ahierarchical structure; and displaying the candidate tags in a mannerindicative of the candidate tags' positions in the hierarchicalstructure.

What is claimed is:
 1. An information processing system comprising: aprocessor for determining one or more candidate tags based on inputdata, the candidate tags being included within a hierarchical structure;and a display for displaying the candidate tags in a manner indicativeof the candidate tags' positions in the hierarchical structure.
 2. Thesystem as recited in claim 1, wherein the processor and display areimplemented in a single device.
 3. The system as recited in claim 1,wherein the system comprises at least one information processing serverand at least one information processing device, the processor beingincluded in the information processing server, and the display beingincluded in the information processing device.
 4. The system as recitedin claim 1, wherein an evaluation value is generated for each candidatetag, the evaluation value being indicative of the candidate tag'scorrespondence with the input data.
 5. The system as recited in claim 4,wherein the candidate tags are displayed in order of descendingevaluation values.
 6. The system as recited in claim 4, wherein thecandidate tags are displayed in a manner that emphasizes selectedcandidate tags according to the evaluation values.
 7. The system asrecited in claim 1, wherein the display simultaneously displays thecandidate tags and the input data.
 8. The system as recited in claim 1,wherein the candidate tags are displayed in a tree format.
 9. The systemas recited in claim 1, wherein the display simultaneously displays thecandidate tags, the input data, and a candidate tag search box.
 10. Thesystem as recited in claim 9, wherein as text is entered in the searchbox fewer candidate tags are displayed.
 11. The system as recited inclaim 1, wherein the processor determines one or more candidate tags bydetermining a process target area of the input data and determining oneor more candidate tags based on the process target area.
 12. The systemas recited in claim 11, wherein the process target area is determinedaccording to a manual input of a user.
 13. The system as recited inclaim 11, wherein the process target area is determined automatically.14. The system as recited in claim 11, wherein the input data is imagedata and the processor performs an image recognition process todetermine the process target area.
 15. The system as recited in claim11, wherein the input data is text data and the processor performs alanguage recognition process to determine the process target area. 16.The system as recited in claim 11, wherein the input data is sound dataand the processor performs a sound recognition process to determine theprocess target area.
 17. The system as recited in claim 1, wherein, foreach displayed candidate tag, a corresponding thumbnail image isdisplayed in the vicinity of the candidate tag.
 18. The system asrecited in claim 1, wherein an object for initiating display ofundisplayed tags is displayed with the displayed candidate tags.
 19. Aninformation processing method comprising: determining one or morecandidate tags based on input data, the candidate tags being includedwithin a hierarchical structure; and displaying the candidate tags in amanner indicative of the candidate tags' positions in the hierarchicalstructure.
 20. A non-transitory computer-readable medium having storedthereon a computer-readable program for implementing an informationprocessing method, the method comprising: determining one or morecandidate tags based on input data, the candidate tags being includedwithin a hierarchical structure; and displaying the candidate tags in amanner indicative of the candidate tags' positions in the hierarchicalstructure.